Cognitive  Engineering:  Toward  a 
Workable  Concept  of  Mind 

It  seems  plain  to  me  now  that  the  “cognitive  revolution”  .  . .  was  a  response  to  the  technologi¬ 
cal  demands  of  the  Post-Industrial  Revolution.  You  cannot  properly  conceive  of  managing  a 
complex  world  of  information  without  a  workable  concept  of  mind. 

— Bruner  (1983,  p.  63) 


Perhaps  no  one  has  understood  the  depth  to  which 
the  ever-increasing  technological  nature  of  the 
human  ecology  has  shaped  psychological  theory 
better  than  Jerome  Bruner.  In  his  memoir  In  Search 
of  Mind  (1983),  Bruner  shared  his  reflections  on 
the  origins  of  the  cognitive  revolution.  Although 
a  great  many  factors  may  have  played  a  role  (e.g., 
Chomsky,  1959;  Miller,  1956;  Newell  &  Simon, 
1972),  Bruner  turns  much  conventional  thinking 
on  its  head,  implying  that  scientists  had  to  invent  a 
theory  of  mind  in  response  to  the  practical  demands 
of  finding  coherent  ways  of  understanding  and 
coordinating  a  largely  invented  world  of  people 
engaged  with  post-industrial  Revolution  technolo¬ 
gies.  The  seeds  of  this  scientific  revolution,  it  seems, 
were  not  so  much  “in  the  air”  as  in  the  digital  cir¬ 
cuitry  and  in  the  need  to  understand  and  manage 
“a  complex  world  of  information.” 

A  Workable  Concept  of  Mind 

The  purpose  of  this  book  is  to  take  additional  steps 
toward  building  what  Bruner  referred  to  as  a  “work¬ 
able  concept  of  mind.”  Special  emphasis  is  given 
here  to  the  word  workable.  The  central  goal  is  to 


provide  methods  and  models  that  can  be  fruitfully 
applied  to  solving  practically  relevant  problems  in 
human-technology  interaction.  These  problems  in¬ 
clude  designing  and  evaluating  technological  inter¬ 
faces,  decision  aids,  alerting  systems,  and  training 
technology,  as  well  as  supporting  human-automa¬ 
tion  interaction  and  human-computer  interaction. 
In  short,  the  aim  of  this  book  is  to  provide  practi¬ 
cal  resources  for  addressing  the  menagerie  of  prob¬ 
lems  making  up  cognitive  engineering  (Hollnagel  & 
Woods,  1983;  Kirlik  &  Bisantz,  1999;  Norman, 
1986;  Rasmussen,  1986).  Along  the  way,  many  con¬ 
tributors  to  this  volume  also  present  insights  and 
approaches  that  may  shed  light  on  fundamental 
problems  in  the  science  of  adaptive  cognition  and 
behavior.  This  may  be  especially  true  when  it  comes 
to  the  challenge  of  understanding  and  formally  ar¬ 
ticulating  the  role  of  the  environment  in  cognitive 
theory. 

Six  themes  unite  the  contributors’  orientation 
toward  developing  a  concept  of  mind  that  is  both 
workable  and  valuable  from  a  cognitive  engineer¬ 
ing  perspective.  These  themes  are  illustrated  in  the 
selection  of  research  problems,  methods,  and  analy¬ 
sis  and  modeling  techniques  presented  in  the  fol¬ 
lowing  chapters. 
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Background  and  Motivation 


1.  An  Ecological  or 
Systems  Perspective 

Pioneers  of  the  cognitive  engineering  discipline, 
such  as  Rasmussen  (1990)  and  Woods  (1995),  have 
emphasized  the  essential  ecological  character  of  the 
cognitive  engineering  enterprise.  What  does  this 
mean,  and  why  is  it  the  case?  The  answers  to  these 
questions  lie  in  the  nature  of  the  systems  studied 
and  the  practical  problems  associated  with  their 
analysis  and  design.  Cognitive  engineering  research 
is  largely  concerned  with  the  analysis  and  design  of 
human-technology  systems  and  the  role  of  technol¬ 
ogy  as  a  mediator  between  humans  and  the  external 
objects  and  events  comprising  a  work  domain.  As 
such,  cognitive  engineering  is  fundamentally  con¬ 
cerned  with  systems  composed  of  humans,  mediat¬ 
ing  technologies  and  task  environments,  as  well  as 
the  interactions  among  these  system  components.  Of 
all  the  schools  or  approaches  to  psychology  relevant 
to  cognitive  engineering  (and  there  are  many),  both 
the  theoretical  orientation  and  unit  of  analysis  pre¬ 
sumed  by  ecological  psychology  are  perhaps  best 
aligned  with  the  task  of  understanding  and  support¬ 
ing  the  mediated  coupling  between  humans  and 
their  environments.  By  reviewing  the  Instructions 
for  Authors  appearing  in  each  issue  of  Ecological 
Psychology ,  one  learns  that  the  focus  of  the  disci¬ 
pline  is  a  broad  range  of  psychological  problems  (per¬ 
ception,  cognition,  communication,  development, 
learning,  etc.),  but  with  one  noteworthy  constraint. 
In  particular,  ecological  psychology  is  concerned  with 
those  problems  only  “to  the  extent  that  those  prob¬ 
lems  derive  from  a  consideration  of  whole  animal- 
environmental  systems,  rather  than  animals  or 
environments  in  isolation  from  each  other.” 

This  broad  and  inclusive  perspective  on  what 
it  means  to  take  an  ecological  approach  to  cogni¬ 
tive  engineering  is  the  one  presumed  in  this  book. 
As  such,  the  approach  is  much  in  the  spirit  of  the 
Joint  Cognitive  Systems  tradition  in  cognitive  sys¬ 
tems  engineering  (Hollnagel,  Mancini,  &  Woods, 
1986).  It  is  inherently  a  systems  approach  in  which 
each  element  of  the  human-technology-environ¬ 
ment  unit  of  analysis  receives  attention  and  treat¬ 
ment  during  analysis  and  design  and  where  each 
element  is  considered  in  light  of  its  functional  role 
within  the  overall  human-technology-environment 
system.  Although  some  authors  in  this  book  use  the 
phrase  “ecological  analysis”  to  highlight  the  study 
of  the  environmental  components  of  these  systems 


and  “cognitive  analysis”  to  highlight  the  study  of  the 
internal  or  mental  activities  of  their  human  com¬ 
ponents,  the  intended  meaning  of  each  term  ■will 
be  clear  from  context.  At  this  point,  suffice  it  to  say 
that  the  perspective  taken  in  this  book  is  that  a  psy¬ 
chological  theory  or  model,  especially  one  capable 
of  providing  cognitive  engineering  with  a  workable 
concept  of  mind,  should  include  a  description  of 
the  “whole”  human-technology-environment  sys¬ 
tem  and  not  merely  internal  cognition. 

This  systems  or  ecological  orientation  toward 
cognitive  engineering  has  long  roots  in  human  fac¬ 
tors,  and  especially  within  the  field  of  “human- 
machine  systems.”  As  noted  by  Sheridan  (2002, 
p.  5),  shortly  after  World  War  II  human  factors  re¬ 
searchers  increasingly  began  to  borrow  modeling 
techniques  from  engineering,  such  as  mathemati¬ 
cal  theories  of  estimation,  information,  decision, 
and  control.  They  did  so  “to  look  at  information, 
control,  and  decision  making  as  a  continuous  pro¬ 
cess  within  a  closed  loop  that  also  included  physi¬ 
cal  subsystems — more  than  just  sets  of  independent 
stimulus-response  relations”  (Sheridan,  2002,  p.  4). 
The  rationale  for  including  a  description  of  physi¬ 
cal  subsystems  within  such  models,  that  is,  descrip¬ 
tions  of  the  task  environment  and  any  mediating 
technology,  was  well  expressed  by  Baron,  a  human- 
machine  systems  engineer:  “Human  behavior,  either 
cognitive  or  psychomotor,  is  too  diverse  to  model 
unless  it  is  sufficiently  constrained  by  the  situation 
or  environment;  however,  when  these  environmen¬ 
tal  constraints  exist,  to  model  behavior  adequately, 
one  must  include  a  model  for  that  environment” 
(Baron,  1984,  p.  6). 

Exactly  this  point,  the  need  for  a  psychological 
model  to  include  an  environmental  model,  was  one 
of  the  defining  features  of  the  theory  and  method 
developed  by  one  of  ecological  psychology’s  pio¬ 
neers,  Egon  Brunswik  (1903-1955).  As  will  be 
discussed  in  greater  detail  by  Goldstein  in  the  fol¬ 
lowing  chapter,  Brunswik  is  perhaps  best  known 
for  his  lens  model,  whose  ecological  nature  is  di¬ 
rectly  apparent  in  its  symmetrical  arrangement  as 
a  pair  of  joined  human  and  environmental  mod¬ 
els  (Brunswik,  1952,  1956;  also  see  Hammond  & 
Stewart,  2001).  The  pervasive  influence  of  Bruns- 
wik’s  theory  of  probabilistic  functionalism  and 
experimental  methodology  of  representative  design 
is  evident  throughout  this  volume. 

This  is  not  to  imply,  of  course,  that  only  Bruns- 
wik’s  ecological  theory  is  capable  of  giving  rise  to 
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workable  resources  for  cognitive  engineering.  For 
example,  Vicente  (1999)  has  drawn  heavily,  and 
profitably  on  the  ecological  theory  of  James  J. 
Gibson  (1979/1986)  in  the  development  of  his 
Cognitive  Work  Analysis  and  Ecological  Interface 
Design  (also  see  Bums  &  Hajdukiewicz,  2004)  cog¬ 
nitive  engineering  techniques.  As  both  Vicente 
(2003)  and  Kirlik  (1995,  2001)  have  observed,  it 
is  possible  to  view  the  ecological  theories  of  Bruns- 
wik  and  Gibson  to  be  complementary  rather  than 
conflicting,  despite  what  much  of  the  psychologi¬ 
cal  research  conducted  in  each  of  the  two  traditions 
may  lead  one  to  believe.  As  such,  one  should  view 
the  research  presented  in  the  current  volume, 
largely  grounded  in  Brunswikian  theory,  and  the 
research  program  of  Vicente  and  his  colleagues, 
influenced  by  Gibsonian  theory,  as  similarly  com¬ 
plementary  rather  than  conflicting.  Neither  sub¬ 
sumes  the  other  with  respect  to  the  central 
problems  addressed  or  the  techniques  provided. 
The  same  could  also  be  said  of  the  distributed  cog¬ 
nition  approach  (e.g.,  Hollan,  Hutchins,  &  Kirsh, 
2000;  Hutchins,  1995)  and  any  other  framework 
embodying  the  ecological  notion  that  the  unit  of 
psychological  analysis  and  modeling  must  span  the 
human-environment  boundary  if  it  is  to  provide 
cognitive  engineering  with  a  workable  concept  of 
mind  (also  see  Clark,  2003;  Dourish,  2001;  Kirlik, 
1998;  Kirsh,  1996;  Olson  &  Olson,  1991;  Zhang 
&  Norman,  1997). 

2.  An  Adaptive, 

Functional  Perspective 

Anyone  who  has  ever  read  (or  even  better,  taught 
from)  Donald  Norman’s  (1988)  insightful  book  The 
Psychology  of  Everyday  Things  will  almost  certainly 
recognize  his  characterization  of  the  problems  faced 
by  a  technology  user  in  terms  of  bridging  a  “gulf  of 
execution”  and  a  “gulf  of  evaluation.”  How  do  I  get 
it  to  work?  What  is  it  doing?  These  are  questions 
we  find  ourselves  asking  all  to  often  in  our  interac¬ 
tions  with  technology.  As  one  who  has  graded  over 
a  thousand  students’  answers  to  “define  and  give  an 
everyday  example”  exam  questions  about  these 
gulfs,  I  have  found  that  if  a  student  is  going  to  get 
anything  correct  on  an  exam  it  is  likely  to  be  these 
questions.  These  concepts  are  immediately  intuitive 
to  anyone  who  has  ever  experienced  difficulty  or 
frustration  when  using  technology,  whether  in  pro¬ 


gramming  a  VCR  or  the  flight  control  automation 
in  a  modem  glass  cockpit  airliner  (Degani,  Shafto, 
&  Kirlik,  1999;  Sarter  &  Woods,  1992). 

One  reason  the  authors  of  this  volume  have 
become  attracted  to  Brunswik’s  functionalist  theory 
of  cognition  and  behavior  is  its  grounding  in  ex¬ 
actly  these  gulfs.  In  particular,  and  as  explained  in 
greater  detail  by  Goldstein  in  chapter  2,  Brunswik’s 
theory  is  founded  in  an  examination  of  the  proxi¬ 
mal-distal  relationships  characterizing  a  person’s 
encounter  with  the  world  (also  see  Tolman  & 
Brunswik,  1935).  A  technological  interface  provides 
us  with  both  proximal,  or  directly  available,  infor¬ 
mation  sources  and  proximal  opportunities  for 
action.  The  intended  target  of  our  interaction,  how¬ 
ever,  is  all  too  often  distal,  or  not  so  directly  avail¬ 
able  to  us:  a  goal  state  to  be  achieved  by  taking  a 
correct  sequence  of  proximal  actions,  and  an  un¬ 
derstanding  of  whether  we  have  achieved  our  goal, 
which  can  often  be  gained  only  by  correctly  inte¬ 
grating  proximally  available  information. 

In  1972,  Newell  and  Simon  began  their  semi¬ 
nal  book,  General  Problem  Solving,  with  an  expres¬ 
sion  of  debt  to  Tolman  &  Brunswik’s  (1935) 
Psychological  Review  article  emphasizing  the  neces¬ 
sity  of  a  detailed  analysis  of  these  proximal-distal 
relations  to  understand  goal-directed  behavior. 
Newell  and  Simon  appreciated  the  insight  that  such 
behavior  is  typically  directed  toward  distal  objects 
or  ends  that  can  be  achieved  only  by  the  adaptive 
use  of  proximal  information  and  action  resources. 
One  result  was  Newell  and  Simon’s  characterization 
of  problem  solving  as  a  search  through  a  “problem 
space”  to  find  a  series  of  proximal  actions  that  would 
lead  to  distal  goals.  A  problem  space  is  a  model  of  a 
problem  solver’s  environment. 

Every  chapter  in  this  volume  deals  in  one  way 
or  another  with  an  examination  of  the  proximal- 
distal  relations  characterizing  or  mediating  one’s 
encounter  with  the  environment  and  a  parallel  ex¬ 
amination  of  the  degree  to  which  humans  are  well 
adapted  to  these  relations.  This  is  what  is  meant 
here  by  an  adaptive,  functional  perspective  on  cog¬ 
nition  and  behavior.  There  is  no  initial  assumption 
that  people  are  either  well  or  poorly  adapted  to  the 
demands  and  opportunities  of  any  particular  situ¬ 
ation.  Instead,  the  approach  taken  in  the  following 
chapters  is  to  perform  detailed  functional  analyses 
of  task  environments  and  then  empirically  measure 
the  degree  of  adaptivity  attained  in  light  of  both  the 
cognitive  and  environmental  resources  available. 
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Various  chapters  in  parts  IV,  V,  and  VI  of  this  book 
also  examine  the  relationship  between  Brunswik’s 
original  approach  to  these  problems  and  more  re¬ 
cent  yet  related  adaptive  approaches  to  cognition 
such  as  Anderson’s  (1990)  Rational  Analysis  and 
ACT-R  model  (Anderson  &  Lebiere,  1998),  and 
the  Ecological  Rationality  approach  developed  by 
Gigerenzer,  Todd,  and  their  colleagues  (Gigerenzer, 
Todd,  &  the  ABC  Research  Group  1999). 

3.  Embracing  Uncertainty 

Another  aspect  of  Brunswik’s  thinking  adopted  by 
the  authors  in  this  volume  is  the  idea  that  the  re¬ 
lationship  between  the  human  and  environment 
must  often  be  characterized  in  probabilistic  terms. 
Note,  however,  that  this  does  not  reflect  an  a  priori 
commitment  to  probabilism  but  instead  the  need 
to  have  conceptual  and  technical  resources  avail¬ 
able  for  measuring  and  modeling  uncertainty  where 
it  is  found  to  exist.  Having  techniques  available  to 
represent  the  possible  probabilistic  structure  both 
within  a  task  environment  and  within  the  opera¬ 
tions  of  inner  cognitive  processes  is  especially  im¬ 
portant  for  the  purpose  of  evaluating  the  adaptivity 
of  behavior,  and  also  when  motivating  interventions 
aimed  at  enhancing  it.  Why? 

First,  environmental  uncertainty  places  a  ceiling 
on  the  accuracy  of  adaptive  behavior  in  any  given 
instance.  As  Iipshitz  et  al.  (2001a)  have  (qualitatively) 
put  it,  “Uncertainty  is  intimately  linked  with  error:  the 
greater  the  uncertainty,  the  greater  the  probability  of 
making  an  error”  (p.  339).  As  such,  it  is  important  to 
recognize  the  existence  of  environmental  uncertainty 
from  a  forensic  perspective  because  human  “error" 
must  always  be  expected  when  people  are  perform¬ 
ing  in  environments  with  irreducible  uncertainty 
(Hammond,  1996).  Second,  the  possible  presence  of 
uncertainty  suggests  that  questions  about  adaptive 
cognition  be  addressed  and  answered  at  the  level  of 
how  well  tailored  or  calibrated  a  performer’s  judg¬ 
ments  or  actions  are  to  the  environment  on  average, 
or  at  a  distributional  level  of  analysis,  rather  than  on 
an  instance-by-instance  basis. 

4.  Embracing  Representativeness 

The  research  presented  in  this  book  shares  with  the 
Naturalistic  Decision  Making  (NDM)  paradigm  the 


goal  of  “conducting  one’s  study  with  representative 
samples  of  subjects,  tasks,  and  contexts  to  which 
one  wishes  to  generalize”  (Lipshitz  et  al.,  2001b, 
p.  386).  This  is  illustrated  in  this  volume  by  the  com¬ 
plete  reliance  on  either  field  observation  or  the  use 
of  dynamic  and  interactive  simulations  modeled 
after  the  target  context  of  scientific  generalization. 
As  will  be  described  by  Goldstein  in  more  detail  in 
chapter  2,  this  methodological  commitment  is  con¬ 
sistent  with  Brunswik’s  methodology  of  represen¬ 
tative  design.  However,  as  will  be  illustrated  in 
nearly  all  of  the  chapters  that  follow,  this  orienta¬ 
tion  does  not  preclude  the  use  of  systematically 
designed  interventions  in  representatively  designed 
experiments.  This  book  is  filled  with  examples  of 
investigators  using  hybrid  representative/systematic 
experimental  designs  to  both  foster  the  generaliza¬ 
tion  of  results  to  a  target  context  and  also  to  test 
various  hypotheses  regarding  the  efficacy  of  design, 
training,  or  aiding  interventions  and  to  examine 
how  adaptivity  may  be  influenced  by  factors  such 
as  time  stress. 

5.  A  Formal  Perspective 

The  research  presented  in  this  volume  displays  a 
commitment  to  abstraction  and  formalization  in  the 
creation  of  modeling  and  measurement  techniques 
(also  see  Byrne  &  Gray,  2003).  The  contributors  to 
this  volume,  as  illustrated  by  their  demonstrated 
commitment  to  study  cognition  and  behavior  in 
context  and  to  perform  formal  (mathematical  or 
computational)  modeling,  agree  with  Todd  and 
Gigerenzer  (2001)  in  noting  that  the  alternatives 
for  describing  context-sensitive,  adaptive  cogni¬ 
tion  “are  not  context-free  formal  modeling  versus 
context-bound  informal  modeling”  (p.  382). 

Instead,  1  hope  that  the  chapters  that  follow 
illustrate  that  it  is  quite  possible  to  have  a  deep 
appreciation  for  the  role  of  the  environmental  con¬ 
text  in  cognition  and  behavior,  yet  also  to  have  an 
appreciation  for  and  ability  to  formally  model  the 
essential  aspects  of  this  context.  As  I  have  pointed 
out  elsewhere  (Kirlik,  2003),  research  in  fields  such 
as  human  factors  and  cognitive  engineering  nearly 
always  begins  (or  should  begin)  with  a  qualitative, 
naturalistic  phase  to  identify  and  distill  the  central 
features  of  a  target  problem  to  be  solved  or  phenom¬ 
ena  to  be  investigated.  Yet  if  attention  then  turns 
directly  to  creating  an  intuitive  solution  or  qualita- 
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tive  account  (regardless  of  how  well  received  by 
stakeholders),  without  bringing  these  central  fea¬ 
tures  to  an  abstract  level,  it  is  often  impossible  to 
know  the  conditions  in  which  that  same  solution 
will  prove  useful.  As  such,  each  cognitive  engineer¬ 
ing  problem  will  have  to  be  solved  largely  from 
scratch.  A  workable  concept  of  mind  useful  for  cog¬ 
nitive  engineering  should  be  one  that  is  fertile  in 
giving  rise  to  a  toolbox  of  formal  analysis  and  mod¬ 
eling  techniques,  as  scientific  generalization  rides 
chiefly  on  the  winds  of  abstraction. 

6.  A  Problem-Solving  Perspective 

The  problem-solving  orientation  displayed  in  the 
research  presented  herein  will  be  immediately  ap¬ 
parent,  What  might  be  less  obvious  in  these  pages 
is  that  in  virtually  every  case  researchers  had  to 
work  with  the  behavioral  situation  given  to  them, 
and  this  required  them  to  invent  a  wide  variety  of 
novel  extensions  and  improvisations  on  the  general 
theoretical  perspective  uniting  these  studies  within 
a  common  intellectual  and  historical  perspective. 
It  is  likely  that  the  more  important  contributions 
within  these  chapters  lie  in  the  development  of  these 
extensions  to  current  theory  and  method,  rather  than 
in  any  particular  empirical  findings  presented.  It  may 
be  useful  to  highlight  this  aspect  of  the  book  at  the 
outset. 

To  explain,  the  study  of  human-technology  in¬ 
teraction  and  the  disciplines  of  cognitive  engineering, 
human  factors,  and  human-computer  interaction  are 
changing  so  rapidly  that  the  practically  relevant  life¬ 
time  of  any  particular  empirical  effect  or  finding  is 
likely  to  be  quite  short.  In  a  world  continually  under¬ 
going  reinvention,  the  specific  barriers  to  adaptive 
cognition  and  behavior  are  likely  to  change  as  rap¬ 
idly  as  the  ecology  itself  (Kirlik,  2005).  As  is  often  the 
case  with  the  workers  and  performers  cognitive  en¬ 
gineers  intend  to  support,  our  own  work  as  research¬ 
ers,  practitioners,  and  students  of  human-technology 
interaction  is  one  of  almost  continuous  adaptation  to 
novel  problems  and  opportunities.  As  such,  a  useful 
method,  model,  or  measurement  technique  is  likely 
to  have  a  longer  shelf  life,  as  measured  by  its  dura¬ 
tion  of  practical  relevance,  than  any  particular  experi¬ 
mental  result  or  field  observation. 

As  such,  one  can  only  hope  that  those  who 
might  embrace  the  perspectives  on  which  this  book 
is  based  and  those  who  might  use  the  methods  and 


models  presented  here  appreciate  that  these  tech¬ 
niques  are  all  works  in  progress,  and  additional 
progress  depends  on  extending  and  elaborating 
these  tools  and  techniques.  Only  then  will  cogni¬ 
tive  engineering  come  to  possess  the  kind  of  diverse 
and  reusable  toolbox  of  measurement,  analysis,  and 
modeling  techniques  characteristic  of  other  engi¬ 
neering  disciplines. 

Conclusion 

After  the  immediately  following  chapter  on  theory 
and  method,  this  volume  consists  of  a  set  of  chap¬ 
ters  presenting  research  adhering  to  each  of  the  six 
general  themes  or  perspectives  outlined  in  this 
chapter.  Although  it  is  traditional  in  a  volume  such 
as  this  to  provide  an  introductory  overview  and 
roadmap  of  the  chapters  that  follow,  this  is  unnec¬ 
essary  for  two  reasons.  First,  in  chapter  2  Goldstein 
provides  an  introduction  and  tutorial  that  explic¬ 
itly  situates  each  chapter  by  name  within  its  intel¬ 
lectual  and  historical  place  in  Brunswik’s  thought 
and  scholarship.  Second,  I  have  provided  introduc¬ 
tory  overviews  prefacing  each  of  the  volume’s  subse¬ 
quent  sections.  Should  one  desire  to  gain  a  quick 
overview  of  the  substantive  contents  of  the  volume, 
reading  Goldstein’s  chapter  and/or  reading  through 
each  of  these  brief  section  introductions  should 
serve  this  purpose  well. 
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